Modern computers provide each process being executed in a computer system its own virtual address space. This allows memory to be dynamically allocated to each running process while at the same time providing all the running processes with contiguous address space. The computer operating system and hardware work together to map each virtual address for each process to a unique physical address thereby making the appropriate location in memory available when needed by a process. The operating system kernel stores this mapping data in page tables in the main memory.
Relocation of data from one physical memory location to another is sometimes necessary. For example, if one memory board begins to fail, it might be desirable to replace it. However, traditionally, for a variety of reasons, virtual-to-physical mapping data has been designated as non-relocatable (NR) and stored along with other non-relocatable data as designated by the operating system.
When a failing memory board contains data that is considered non-relocatable (NR), special measures must be taken. Often, the system is simply shut down, the memory board swapped out and the system restarted. This is undesirable for obvious reasons, such as the down-time of the system. NR data could also be relocated using a hardware based copy-rename algorithm. In this case, all devices attached to the system are suspended, all CPUs are paused, the entire memory is copied from the old source board to the new destination board, and the physical addresses on the new board are reprogrammed to be the same as those on the old board which is to be removed. The old board is then deactivated and the system resumed.
Hardware copy-rename has severe limitations. First, since it pauses all the remaining CPUs and suspends all devices, system performance is drastically affected. Thus, it is only used while removing the board on which operating system (OS) kernel memory is placed. Second, copy-rename requires all the kernel memory including the kernel/user stacks to be allocated on the minimum number of boards. Otherwise, it may not be possible to reprogram the new board with the physical addresses of the old board. All NR data is therefore allocated on the minimum number of boards. As a result, threads running on all CPUs access the same memory board(s), which may be remote from the CPUs, whenever the mapping data is accessed. This increases memory latency and bus traffic and thus affects the scalability of large servers.
It would be desirable to provide a mechanism to relocate virtual-to-physical mapping data without significantly affecting system availability. This would allow the mapping data to be initialized at a physical location closer to the processor requesting memory, which would improve memory latency while servicing a translation lookaside buffer (TLB) miss or pagefault. In addition, it would reduce the amount of NR memory required by the kernel. In addition, by not having to allocate static buffers in the NR memory for holding the virtual-to-physical mappings during boot, initialization time would be decreased.